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Chapter  1 


INTRODUCTION 

Statement  of  the  Problem 


As  the  complexity  of  weapons  systems  increased 
during  the  1960s,  it  became  more  difficult  to  accurately 
estimate  the  related  acquisition  costs,  both  in  the  early 
development  and  production  phases  of  the  acquisition  cycle 
(8:7).  Faulty  cost  estimates,  coupled  with  inflation, 
were  blamed  as  causing  numerous  projects  to  experience 
significant  cost  growth  and  schedule  delays  (8:7)  . In 
response  to  this  problem,  the  Department  of  Defense  (DOD) 
initiated  several  activities  designed  to  improve  its  cost 
estimating  capabilities  for  Government  contracts.  One  of 
the  approaches  developed  was  a concept  called  Should  Cost 
To  establish  a common  baseline  for  departure  and 
deal  in  more  specific  terms,  the  following  definition  of 
Should  Cost  which  was  developed  by  the  U.S.  Air  Force 
(.22:1-1)  is  set  forth: 

Should  Cost.  A technique  of  contract  pricing 
that  employs  an  integrated  team  of  Government  pro- 
curement, contract  administration,  audit,  and 
engineering  representatives  to  conduct  a coordinated, 
in-depth  cost  analysis  at  the  contractor's/sub- 
contractor's plants.  The  objective  is  to  identify 
uneconomical  or  inefficient  practices  in  the  contrac- 
tor's management  and  operations  and  to  quantify  the 
findings  in  terms  of  their  impact  on  cost.  The 
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result  is  the  development  of  a realistic  price  objec- 
tive which  reflects  reasonably  achievable  economies 
and  efficiencies. 

A total  of  twenty-three  Should  Cost  studies  were 
conducted  by  ASD  during  the  calendar  year  period  1971 
through  1974  (3:3).  These  twenty-three  studies  demon- 
strate that  Should  Cost  has  been  used  discriminantly  by 
ASD  as  a cost  analysis/negotiation  tool  (3:3).  The  results 
of  these  Should  Cost  reviews  have  proven  beneficial  to 
the  extent  that  they  have  supported  negotiation  positions 
maintained  by  Air  Force  contract  negotiators.  A survey 
of  articles  and  research  works  pertaining  to  Should  Cost 
indicates  that  the  literature  deals  primarily  with  the 
methodological  and  organizational  aspects  of  applying  the 
Should  Cost  technique  in  the  procurement  environment. 

Many  of  these  studies  presuppose  that  the  application  of 
the  Should  Cost  technique  produces  lasting  contractual 
cost  savings  to  the  Government.  While  offering  some  valu- 
able insights  into  the  usefulness  of  Should  Cost,  the 
literature  at  the  same  time  generally  overlooks  the  vital 
matter  of  the  validity  of  cost  savings  as  highlighted  by 
Wayne  M.  Allen  (1:48). 

The  Should  Cost  philosophy  has  the  admirable  goal 
of  obtaining  for  the  Government  optimal  efficiency 
in  contractor  operations.  However,  as  a practical 
matter  there  are  some  difficulties.  The  buyer  is  pre- 
pared to  tell  the  seller  how  the  sellar  snould  conduct 
his  business.  . . . While  this  is  an  oversimplifica- 
tion, the  fact  remains  that  under  Cost  Plus  contrac- 
ting procedures  the  buyer  generally  assumes  responsi- 
bility for  most  of  the  costs.  Thus,  the  claims  we 
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are  hearing  today  for  cost  savings  arising  from  Should 
Cost  studies  may  be  premature.  It  remains  to  be  seen 
whether  costs  ’saved*  by  virtue  of  such  studies  stay 
saved  over  the  life  of  the  procurement. 

In  summary,  the  initial  results  of  Should  Cost 
studies  have  shown  that  the  application  of  the  Should  Cost 
analysis  technique  does  offer  potential  but  unproven  con- 
tract cost  savings  (3:3).  The  problem  then,  is  whether 
the  contract  cost  savings  are  real  and  lasting  or  whether 
they  are  a short-lived  phenomena  manifested  by  lowered 
negotiated  contract  cost  targets,  only  to  find  the  final 
contract  costs  or  profits  paid  by  the  Government  to  be 
higher  than  those  originally  contemplated. 


Significance  of  the  Problem 

The  continued  application  of  the  Should  Cost  tech- 
nique without  a sound  assessment  of  its  effectiveness  can 
have  an  adverse  effect  upon  the  Air  Force's  stated  policy 
of  selected  usage  (22:1-1).  The  Should  Cost  concept  as 
employed  by  the  Air  Force  may  require  substantial  personnel 
and  travel  expense;  thus,  unwise  Should  Cost  application 
could  result  in  dissipation  of  critical  personnel  resources 
and  the  incurrence  of  unwarranted  costs  (22:1-1). 


Background 


Should  Cost  explained.  Cost  analysis  is  an  inherent  element 
of  the  Should  Cost  technique.  The  Armed  Services  Procure- 
ment Regulation  (ASPR)  defines  cost  analysis,  as  it  relates 


to  Government  procurement  transactions,  as  . 


(21:13) : 
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. . . the  review  and  evaluation  of  a contractor's 
cost  or  pricing  data  and  of  the  judgemental  factors 
applied  in  projecting  from  the  data  to  the  estimated 
costs,  in  order  to  form  an  opinion  on  the  degree  to 
which  the  contractor's  proposed  costs  represent  what 
performance  of  the  contract  should  cost,  assuming 
reasonable  economy  and  efficiency. 

Some  of  the  present  day  methods  of  cost  analysis  are: 

1.  The  historic  method  where  the  estimate  is 
based  on  previous  procurements  of  similar  items; 

2.  The  engineering  or  building  block  approach 
which  entails  an  examination  of  separate  items  of 
work  at  a low  level  of  the  work  breakdown  structure 
with  detailed  estimates  developed  for  functional 
costs  of  engineering,  manufacturing,  quality  control, 
etc.,  which  are  in  turn  sub-categorized  by  labor, 
material,  and  other  cost  estimates; 

3.  The  parametric  costing  technique  predicts 
costs  by  means  of  explanatory  variables  such  as  per- 
formance characteristics,  physical  characteristics, 
and  characteristics  relevant  to  the  development 
process  as  derived  from  experience  on  related  sys- 
tems; and 

4.  The  Should  Cost  approach  which  seeks  to 
develop  a realistic  price  objective  by  quantifying 
the  impact  on  cost  of  inefficient  or  uneconomical 
practices  in  the  contractor's  management  and  opera- 
tions. 

This  Should  Cost  concept  is  not  new.  It  is,  or 
should  be,  an  inherent  aspect  of  each  contract  cost 
analysis  performed.  What  is  relatively  new  is  the  manner 
in  which  the  concept  is  applied  using  highly  specialized 
teams  of  individuals  (either  independent  consultants  and/or 
DOD  employees)  to  perform  the  in-depth  analyses  of  con- 
tractors' operations  (22:1-1).  A Should  Cost  review 
considers  all  activity  in  a contractor's  plant;  however. 
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usually  it  is  conducted  in  connection  with  the  cost 
analysis  of  one  program  or  product. 

Should  cost  differs  from  traditional  pricing  pri- 
marily in  two  respects:  (1)  depth  of  analysis;  and 

(2)  extent  to  which  the  Government  challenges  inefficien- 
cies in  contractor  operations  (16:20).  Traditional 
pricing  efforts  are  based  largely  on  projections  of  his- 
torical cost  data.  When  past  operations  were  inefficient, 
such  projections  of  cost  data  included  an  automatic  "built- 
in"  factor  covering  the  cost  of  continued  inefficiency 
(11:3).  Through  careful  analysis,  these  inefficient  prac- 
tices can  be  discovered  in  contractors'  current  proposals 
(11:4).  The  Air  Force  can  then  be  reasonably  assured  that 
only  those  costs  which  would  be  incurred  under  conditions 
of  economy  and  efficiency  of  operations  will  be  incurred 
(11:4). 

Objectives  of  Should  Cost.  The  Government  hopes  to 
accomplish  two  important  objectives  by  performing  Should 
Cost  analyses  (11:4).  The  first  is  to  establish  a Govern- 
ment negotiation  position  (11:4).  Corollary  to  this  goal 
is  the  hoped-for  short-term  benefits  which  are  those 
savings  to  be  accrued  by  the  Government  during  the  instant 
contractor's  period  of  performance  (22:2-2).  It  should  be 
emphasized  that  the  Should  Cost  figure  may  not  be  one  which 
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is  attainable,  but  one  which  reflects  the  dollar  effect 
of  achievable  economies  and  efficiencies  (11:4). 

The  second  objective  is  to  encourage  and  assist 
the  contractors  in  becoming  more  efficient  (16:23).  This 
benefits  not  only  the  Government,  but  the  contractor  as 
well,  since  it  improves  the  contractor's  ability  to  com- 
pete (15:23).  The  long-term  benefits  to  the  Government 
are  the  potential  savings  resulting  from  the  contractor's 
management  improvements  which  could  apply  either  to  follow- 
on  procurements  or  to  other  product  lines  (22:2-2). 

Commitments  are  obtained  from  the  contractor's 
top  mancoement  to  correct  specific  inefficient  practices 
which  are  identified  through  Should  Cost  reviews  (15:21). 

A few  of  the  specific  management  areas  which  have  been 
brought  to  contractors'  attention  as  a result  of  past 
Should  Cost  reviews  are:  (1)  improve  or  establish  direct 

labor  standards  ; (2)  reduce  scrap;  (3)  improve  a make- 
or-buy  system;  (4)  improve  plant  layout;  and  (5)  improve 
production  management  (11:5). 

Cased  on  experience  to  date.  Should  Cost  offers 
a noteworthy  capability  for  obtaining  a better  understand- 
ing of  the  contractor's  operation  which,  in  turn,  can  be 
translated  into  a more  comprehensive  evaluation  of  costs 
proposed  to  the  Government  (22:1-1).  The  Should  Cost 
approach,  as  used  by  the  Government,  is  not  viewed  as  a 
cure-all  to  "cost  growth"  or  "cost  overruns. 


To  avoid 
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confusion,  these  terms  will  be  explained  at  this  point 
(5:364-365) : 

Cost  overrun  is  that  sum  of  money  spent  on  a 
program  in  excess  of  contract  price. 

Cost  growth  is  the  increase  in  program  expendi- 
tures above  the  price  of  the  original  program 
plan.  *"  ' 

However,  Should  Cost  is  viewed  as  a potent  analytical  tool 
for  use  in  those  cases  which  meet  the  Should  Cost  applica- 
tion criteria  (22:1-1). 

Criteria  for  Should  Cost  application.  The  Air  Force  posi- 
tion states  the  Should  Cost  technique  should  be  used  in 
those  selective  instances  where  the  Air  Force  can  antici- 
pate a major  payoff  by  its  application  (20:2-1).  The  Air 
Force  has  found,  ideally,  that  Should  Cost  reviews  are 
most  productive  when  employed  cn  procurements:  (1)  for 

which  there  are  future  year  production  requirements  for 
substantial  quantities  of  like  items;  (2)  there  has 
already  been  some  production — in  all  probability,  the 
initial  production  run;  (3)  a sole-source  situation  exists; 

(4)  the  specifications  are  comparatively  definitive;  and 

(5)  the  present  and  potential  value  of  the  work  is  sub- 
stantial (22:2-1).  It  is  Air  Force  policy  that  virtually 
all  situations  involving  continuing  procurement  of  items 
from  a sole-source  contractor  should  be  considered  as 


candidates  for  new  Should  Cost  studies  (22:2-1). 
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reviews  (22:2-2): 

1.  Major  On-Going  System.  Optimum  results  should 
occur  when  the  contractor's  procedures  for  quality  pro- 
duction have  been  established.  Under  these  circumstances, 
inefficiencies  and  uneconomical  practices  may  be  identi- 
fied and  the  impact  of  their  correction  measured. 

2.  Imminent  Follow-on  Buy.  An  imminent  follow-on 
buy  may  exist  which  will  provide  opportunity  for  correc- 
tion of  inefficiencies  ard  uneconomical  practices  poten- 
tially resulting  in  future  cost  reductions. 

3.  Cost  Effectiveness.  The  relationship  between 
the  potential  savings  which  may  accrue  to  the  Government 
and  the  cost  of  conducting  a Should  Cost  analysis  is  of 
major  importance.  The  possibility  for  savings  should  be 
commensurate  with  the  effort  involved. 

4.  History  of  Increasing  Costs.  Trends  of 
increasing  costs  for  the  acquisition  of  the  same  system, 
subsystem,  or  component  generally  are  indicative  of  a 
situation  where  Should  Cost  can  be  used  productively. 

5.  Preponderance  of  Government  Business.  Where 
the  contractor's  business  is  predominantly  Government 
rather  than  commercial,  competitive  forces  may  not  be 
significant  enough  to  encourage  good  cost  controls. 
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The  appropriate  timing  for  the  application  of  a 
Should  Cost  review  is  debatable , but,  since  the  technique 
is  highly  product-oriented,  it  would  probably  become  more 
justified  as  the  product  advances  through  the  system  life 
cycle  and  the  number  of  uncertainties  decreases. 

Elements  of  analysis.  The  elements  of  analysis  encompassed 
in  the  Should  Cost  study  are  directed  at  a quantitative 
evaluation  of  the  contractor ' s proposal  and  operations 
(10:5-1).  The  analysis  of  the  contractor's  manufacturing 
operations  is  vital  to  the  success  of  the  Should  Cost 
review  as  it  represents  the  realistic  baseline  upon  which 
the  ultimate  cost  objective  is  developed  (22:5-1).  There- 
fore, the  depth  and  extent  of  analysis  of  the  manufacturing 
operations  is  carefully  planned  and  monitored  to  ensure 
results  which  apply  to  the  specific  contract  under  evalua- 
tion and  effective  use  of  the  team  members  (22:5-1).  The 
objective  of  the  manufacturing  operations  evaluation  is 
to  identify  uneconomic  or  inefficient  practices  and  quan- 
tify the  findings  in  terms  of  their  impact  on  cost  so  as 
to  develop  ?.  realistic  price  objective  for  negotiation 
(22:5-1).  The  negotiation  objective  is  a price  at  which 
it  is  believed/predicted  that  the  contractor  can  produce 
if  he  operates  with  reasonable  economy  and  efficiency 
(11:5).  The  Government  does  not  seek  to  tell  a contractor 
how  to  run  his  business.  It  does,  however,  have  a 
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commitment  to  the  American  taxpayer  not  to  pay  for  ineffi- 
cient practices  in  the  acquisition  of  major  weapons  systems 
(11: 5) . 

previous  Studies 

Since  1970,  the  popularity  of  Should  Cost  has 
grown  significantly  and  numerous  articles  and  research 
works  on  the  subject  of  Should  Cost  have  been  published. 

A review  of  this  literature  by  the  researchers  revealed 
that  it  dealt  generally  with  the  application  of  the  "how 
t .c"  and  "by  whom"  mechanics  of  the  Should  Cost  technique 
or  suggesting  another  way  of  applying  it.  There  was  evi- 
dence of  quantitative  research  into  the  cost  results  of 
Should  Cost  studies.  However,  the  scope  of  the  research 
was  limited  to  quantitative  analyses  of  the  comparisons 
of  proposed  costs  and  negotiated  costs  showing  the 
resultant  improvements  in  negotiation  position  to  validate 
the  application  of  the  Should  Cost  technique.  This  limita- 
tion in  scope  can  be  attributed  to  the  state  of  available 
cost  data  at  the  time  of  the  research. 

As  the  work  under  contracts  negotiated  using 
Should  Cost  nears  completion,  data  relating  to  actual  con- 
tract costs  becomes  available.  It  is  believed  that  a more 
in-depth  quantitative  analysis  of  the  impact  of  Should  Cost 
on  costs  over  the  life  of  the  contract  is  in  order. 
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Objective 

The  primary  objective  of  this  research  study  was 
to  quantify  the  comparison  of  contract  cost  outcomes  on 
Should  Cost  negotiated  contracts  with  those  contracts 
negotiated  using  conventional  cost  analysis  techniques. 


Scope 

This  research  effort  was  concerned  with  Air  Force 
conducted  Should  Cost  reviews.  The  study  was  confined  to 
the  Aeronautical  Systems  Division  (ASD)  of  the  Air  Force 
Systems  Command. 

As  mentioned  earlier,  the  primary  objective  of 
this  research  study  was  to  quantify  the  comparison  of  con- 
tract cost  outcomes  on  Should  Cost  negotiated  contracts 
versus  those  negotiated  using  conventional  cost  analysis 
techniques.  To  arcomplish  this  objective,  the  criteria 
examined  were  limited  to  cost  data  elements  which  were 
quantitatively  measurable.  Those  qualitative  criteria 
such  as  long  range  management  improvement  recommendations, 
which  generally  can  only  be  indirectly  measured,  were 
excluded  and  are  addressed  in  the  Summary  List  of  Assump- 
tions/Limitations . 


Research  Hypothesis 

There  is  a difference  between  the  actual  cost  out- 
comes on  contracts  negotiated  using  Should  Cost  methods  and 


Chapter  2 

SHOULD  COST  STUDIES: 

AERONAUTICAL  SYSTEMS  DIVISION 

The  content  of  this  chapter  is  intended  to  provide 
a general  understanding  of  the  environment , conditions, 
and  criteria  underlying  Should  Cost  activity  in  the  Aero- 
nautical Systems  Division  (ASD)  and  to  set  the  stage  for 
the  reader's  understanding  of  the  research  performed,  the 
results  of  which  are  presented  in  the  following  chapters. 

The  Aeronautical  Systems  Division  of  the  Air  Force 
Systems  Command  (AFSC)  plans  and  manages  the  acquisition 
of  aeronautical  systems,  subsystems,  and  associated 
equipment  (4:1).  ASD  is  directly  responsible  to  Head- 
quarters AFSC  for  mission  accomplishment.  Within  this 
mission  responsibility,  ASD  enters  into  contracts  and  man- 
ages all  phases  of  procurement  and  production  in  support 
of  such  diverse  products  as  major  weapon  systems,  research 
and  development,  services,  Aerospace  Ground  Equipment 
(AGE),  aircraft  engines  and  supplies  (4:1).  ASD  is 
organized  on  a functional  and  projectized  basis  to  per- 
form this  mission  responsibility.  The  Directorate  of 
Pricing  within  the  Deputate  of  Procurement  and  Production 
has  the  primary  responsibility  for  managing  Should  Cost 
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activities  in  the  ASD  organization.  ASD  has  been  very 
selective  in  the  application  of  Should  Cost  (4).  This 
selective  application  is  in  consonance  with  the  overall 
Mr  Force  policy  which  advocates  selective  usage  of  the 
technique,  thus  avoiding  excessive  and  disruptive  expendi- 
tures of  manpower  and  personnel  resources  (22:1-1).  The 
physical  performance  of  the  Should  Cost  review  by  ASD  is 
conducted  by  a team  which  is  structured  to  meet  the  needs 
of  a specific  procurement  (4).  The  selection  of  team 
members  is  critical  to  the  success  of  the  activity;  there- 
fore, ASD  strives  to  achieve  a balance  of  disciplines 
on  each  team  to  assure  a broad,  but  adequate  coverage  of 
functional  activities.  The  team  chief,  who  generally  is 
a senior  management  individual,  is  responsible  for  the 
overall  Should  Cost  operation  from  the  planning  phase  to 
final  contract  negotiation  and  documentation  (4) . The 
procuring  contracting  officer  (PCO)  serves  in  his  normal 
contracting  role  while  the  price  analyst  operates  norma) ly 
as  the  chief  negotiator  (4). 

In  performing  the  Should  Cost  studies,  ASD  has  the 
objective  of  producing  significant  efficiencies  and  dollar 
savings  (4).  ASD  views  the  Should  Cost  technique  as  an 
extension  of  the  cost  analysis  method  for  analyzing  con- 
tractor cost  proposals  (4).  This  extended  cost  analysis 
concept  involves  a greater  than  normal  analysis  of  the  pro- 
posal cost  elements;  however,  an  in-depth  industrial 


15 

engineering  evaluation  of  the  processes  which  drive  the 
cost  incurrence  actions  may  not  be  made.  It  can  be 
argued  that  normal  cost  analysis  may  provide  the  same 
benefits  available  through  Should  Cost.  ASD,  however, 
believed  that  the  insights  gained  from  the  in-depth  cost 
analysis  reviews  as  performed  by  them  could  achieve  nego- 
tiated reductions  comparable  to  those  performed  under  a 
highly  structured  Should  Cost  review  which  relies  on  an 
industrial  engineering  approach  to  the  analysis  (4) , and 
better  thai  that  achieved  from  a normal  cost  analysis. 

As  previously  mentioned,  ASD  performed  twenty- 
three  Should  Cost  studies  during  the  calendar  year  period 
April  1971  through  1974.  As  a means  for  evaluating  the 
effectiveness  of  these  studies  where  Should  Cost  was  used 
versus  cases  where  normal  cost  analysis  was  used,  ASD 
made  a study,  the  results  of  which  are  shown  in  Table  1. 
Rather  than  discuss  the  details  of  the  data  in  the  table, 
emphasis  is  placed  on  ASD's  ability  to  sustain  67.5  per- 
cent of  a proportionately  larger  (24.4  percent)  proposed 
reduction  amount.  Although  the  67.5  percent  is  less  than 
the  73.9  percent  sustained  reduction  as  a result  of 
normal  cost  analysis,  the  dollar  return  potential  under 
Should  Cost  is  much  greater  due  to  the  higher  proposed 
reduction  amount  of  $149.9  million.  It  should  be  noted 
that  the  contractors  selected  by  ASD  for  the  comparative 
review  were  not  comprised  of  identical  contractors  as 


PiPPPPlifP^ 


17 

those  included  in  the  Should  Cost  sample  and  those  in  the 
normal  cost  analysis  sample.  The  different  mix  of  con- 
tractors, reviewed  in  this  study,  could  have  biased  the 
data  statistically  resulting  in  a distortion  of  proposed 
reductions.  These  reductions  were  24.4  percent  and  8.3 
percent  respectively  for  those  analyses  where  Should  Cost 
and  normal  cost  analysis  were  used.  The  ASD  statistical 
data  was  mentioned  above  principally  to  establish  one  con- 
venient baseline  from  which  ASD  measured  its  Should  Cost 
study  effectiveness.  The  data  was  not  used  during  this 
research  other  than  to  familiarize  the  reader  with  the 
scope  of  the  ASD  Should  Cost  activity.  Accordingly, 
further  analysis  to  establish  the  validity  or  nonvalidity 
of  the  ASD  data  was  not  made  herein. 

As  previously  stated,  ASD  conducts  its  Should 
Cost  studies  as  an  extension  of  the  cost  analysis  method 
for  analyzing  contractor  cost  proposals  (4).  There  is 
opportunity  cost  associated  with  achieving  the  benefits 
provided  through  the  extended  cost  analysis  efforts.  This 
"extended"  approach,  based  on  the  ASD  experience,  repre- 
sented the  deployment  of  as  few  as  three  skilled  special- 
ists to  as  many  as  nineteen.  The  labor  and  related  sup- 
portive costs  associated  with  the  twenty-three  ASD  Should 
Cost  studios  are  shown  on  Table  2.  This  table  has  been  pre- 
sented merely  to  reflect  some  historical  data  which  may 
give  the  reader  of  this  thesis  an  appreciation  for  the 
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additional  cost  involved  in  conducting  the  ASD  Should  Cost 
program.  The  costs  are  displayed  to  reflect  those  addi- 
tive costs  associated  with  the  "extended"  portion  of  the 
cost  analysis  effort  and  those  "sunk"  costs  associated  with 
the  normal  cost  analysis.  Although  no  detailed  analysis 
was  made  to  ascertain  the  validity  of  the  cost  data,  a cur- 
sory analysis  was  made  by  the  researchers  to  explain  the 
decreasing  average  cost,  by  year,  for  the  Should  Cost  stud- 
ies conducted.  Basically,  the  decreasing  average  cost  could 
be  attributed  to  the  fact  that  nine  follow-on  studies  were 
conducted  in  fiscal  years  1973,  1974,  and  1975,  and  for 
each  study,  there  were  fewer  team  members  then  on  the  pre- 
vious study,  thus  the  reduced  salary  and  travel  costs. 

ASD  supports  the  precept  that  Should  Cost  is  a 
joint  responsibility  of  the  Procuring  Contracting  Officer 
(PCO)  and  the  Administrative  Contracting  Officer  (ACO)  (8). 
In  this  regard,  it  is  recognized  that  Should  Cost  reviews 
most  generally  will  produce  not  only  short-term  findings 
which  affect  current  negotiations  of  an  instant  procurement 
but  will  produce  management  and  production  improvements 
that  will  result  in  long-term  economy  of  operations  and 
future  cost  avoidance  (22:7-2).  Accordingly,  ASD 
encourages  each  PCO  to  advise  the  responsible  contractor 
and  ACO,  by  letter,  of  the  recommendations  and  request 
that  appropriate  action  be  taken  to  assure  corrective 
action  and  continued  surveillance  (9) . 
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Chapter  3 
METHODOLOGY 

The  material  presented  in  this  chapter  describes 
and  explains  the  logic  used  to  develop  the  statistical 
tests  presented  in  Chapter  4.  The  discussion  centers 
around  five  main  sections:  testing  of  the  research 

hypothesis;  preliminary  data  investigation;  definition 
and  measurement  of  a population;  data  collection;  and  list- 
ing of  assumptions  and  limitations. 

Testing  the  Research  Hypothesis 

The  research  hypothesis  to  be  tested  was: 

There  is  a difference  between  the  actual  cost  out- 
comes on  contracts  negotiated  using  Should  Cost  methods 
and  the  actual  cost  outcomes  on  those  contracts  negotiated 
using  conventional  cost  analysis  techniques. 

Here  the  question  to  be  analyzed  was  whether  the  differences 
between  the  samples  of  Should  Cost  contracts  and  conven- 
tional cost  analysis  contracts,  in  cost  outcomes  at  various 
stages  of  contract  activity,  were  statistically  "signifi- 
cant." In  this  case,  statistical  significance  means  that 
enough  data  have  been  collected  and  analyzed  in  the  sample 
to  establish  that  differences  do  exist  in  the  cost  outcomes 
between  the  populations  of  Should  Cost  and  non-Should  Cost 
contracts. 
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Classical  hypothesis  testing  methods  were  used 
to  test  the  research  hypothesis.  Statistical  hypotheses 
were  formulated  from  the  stated  research  hypothesis.  The 
basic  assumption  or  hypothesis  was  that  there  is  no  differ- 
ence between  the  cost  outcomes  of  the  two  populations  of 
contracts.  This  hypothesis  is  called  the  "null  hypothesis" 
(Hq)  and  is  the  basic  hypothesis  to  be  tested.  The 
reseach  hypothesis  or  the  assumed  condition  that  there  is 
a difference  between  the  two  contract  populations  is 
referred  to  as  the  " alternate " hypothesis  (H^) . If  the 
statistical  hypothesis  testing  determines  that  the  null 
hypothesis  cannot  be  rejected,  then  the  research  hypothesis 
is  not  supported  by  the  data.  Likewise,  if  the  testing 
determines  that  the  null  hypothesis  can  be  rejected,  then 
the  research  hypothesis  .is  supported  by  the  data  at  the 
particular  level  of  testing. 

The  statistical  hypothesis  testing  was  accomplished 
by  comparing  the  differences  of  like  data  elements  (cost 
outcomes  at  various  stages  of  contract  activity)  in  the 
two  sample  contract  groups  (paired  by  contractor)  to  deter- 
mine if  differences  were  significant  so  that  a decision 
could  be  made  to  either  reject  (statistically  significant) 
or  not  reject  (not  statistically  significant)  the  null 
hypothesis) . 

The  decision  to  reject  or  not  reject  the  null 
hypothesis  was  based  on  treating  the  differences  between 


;7^,-  ---z-i"?  ^ *^-  ■ -.  -■  -- 


22 

like  data  elements  in  the  two  paired  contract  sample 
groups  as  a single  sample  and  calculating  the  average 
difference  (6)  of  each  sample.  The  D was  then  used  to 
construct  a ninety-five  percent  confidence  interval  for 
the  average  difference  (A)  between  the  same  data  elements 
in  the  population  of  contracts  from  which  the  sample  was 
taken.  The  formula  used  to  compute  the  confidence  interval 
was  as  follows  (23:172): 


C.I. 


5 + t 


025 


where  D is  the  mean  or  average  difference  between  like 
data  elements  in  each  of  the  two  paired  sample  groups; 
t 025  is  the  critical  value  pertaining  to  the  upper  and 
lower  tail  of  the  "t"  distribution  and  obtained  from  an 
appropriate  statistical  table  of  values;  SD  is  the  standard 
deviation  of  the  individual  differences  from  5;  and  n is 
the  number  of  observations  in  each  sample. 

The  confidence  interval  is  a ranqe  of  values  and 
is  an  interval  estimate  of  the  population  parameter  A 
using  the  sample  statistic  5 and  making  an  allowance  for 
sampling  error. 

The  statistical  null  hypothesis  was  that  there  is 
no  difference  in  cost  outcomes  between  the  two  contract 
population  groups;  i.e.,  A would  be  equal  to  zero,  indi- 
cating no  difference  existed.  The  alternate  hypothesis 
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stated  that  there  ij>  a difference  between  the  cost  out- 
comes of  the  two  groups,  thus  A would  not  equal  zero, 
indicating  either  a positive  or  negative  difference 
existed.  The  null  and  alternative  hypotheses  can  be 
symbolized  as  follows: 

Null  Hypothesis  HQ:  A = 0 
Alternate  Hypothesis  H^ : A ^ 0 . 

If  the  confidence  interval  computed  for  the  average 
difference  in  the  population  (A)  encompasses  zero,  the 
the  hypothesized  value  for  A,  as  graphically  represented 
below,  then  the  null  hypothesis  cannot  be  rejected.  It 
could  be  concluded  from  the  sample  that  the  population 
average  difference  was  zero  indicating  no  difference 
between  the  two  groups  of  contracts  (Should  Cost  versus 
non- Should  Cost) . Therefore,  the  research  hypothesis 
would  not  be  supported. 

Average  Difference  (A)  -25  0 +25 

in  the  Population ^ I > 

Confidence  Interval 
CANNOT  REJECT  NULL  HYPOTHESIS 

If  the  confidence  interval  for  A excludes  the  hypothesized 
value  of  zero,  as  shown  below,  then  the  null  hypothesis 
can  be  rejected.  Therefore,  the  research  hypothesis  would 
be  supported. 
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Average  Difference  (A)  0 +25  +30 

in  the  Population | ^ ^ 
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Confidence  Interval 
CAN  REJECT  NULL  HYPOTHESIS 

Hypothesis  testing  may  be  conducted  at  varying 
levels  of  risk.  The  ninety-five  percent  confidence  level 
(or  five  percent  significance  level)  was  arbitrarily  chosen 
by  the  researchers  to  test  the  null  hypothesis. 

Preliminary  Investigation 

The  previously  stated  research  hypothesis  to  be 
tested  was: 

There  is  a difference  between  the  actual  cost  out- 
come on  contracts  negotiated  using  Should  Cost  methods  and 
the  actual  cost  outcome  on  those  contracts  negotiated 
using  conventional  cost  analysis  techniques. 

To  evaluate  this  hypothesis,  a preliminary  investigation 
was  conducted  at  the  Aeronautical  Systems  Division  (AsD), 
Air  Force  Systems  Command,  Wright-Patterson  Air  Force  Base, 
Ohio.  This  preliminary  investigation  was  concerned  pri- 
marily with  the  availability  of  data  for  use  during  the 
later  research  work.  The  initially  conceived  approach  for 
the  preliminary  investigation  was  to  make  a comparative 
analysis  of  completed  contract  costs  incurred  under 
contracts  of  the  same  general  scope  and  product  class, 
with  the  same  contractor,  which  were  negotiated  using 
Should  Cost  and  completed  costs  of  those  contracts  nego- 
tiated by  using  conventional  cost  analysis  techniques. 
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During  this  preliminary  investigation,  it  was  learned  that 
initial  negotiated  contract  cost  data  were  available  within 
the  ASD  Directorate  of  Pricing  Office  and  that  final  con- 
tract costs  were  available  from  other  sources.  Those 
sources  included  the  ASD  System  Program  Offices  as  well 
as  both  Government  and  contractor-prepared  contract  cost 
reports.  The  investigation  also  indicated  that  ASD,  during 
the  calendar  year  period  1971  through  1974,  had  conducted 
twenty-three  Should  Cost  studies.  Of  these  twenty- three 
contracts,  the  two  contracts  listed  below  were  selected  to 
be  included  in  the  preliminary  investigation.  These  two 
contracts  were  selected  because  the  data  available  at  the 
time  indicated  that  the  contract  work  was  approaching 
completion,  hence  a situation  where  the  desired  data  should 
be  available  to  the  researchers. 

Contractor  (Coded)  Type  Contract  Fiscal  Year 

# 3 Fixed  Price  Incentive  FY72 

#6  Fixed  Price  Incentive  FY73 

As  a basis  for  comparison  of  the  above  Should  Cost  negotia- 
ted contract  costs  versus  those  negotiated  using  conven- 
tional cost  analysis  techniques,  two  contracts  negotiated 
using  conventional  cost  analysis  techniques  during  fiscal 
years  1971  and  1972  respectively,  with  the  same  companies, 
and  for  the  same  products  (missiles) , were  selected  for 
comparative  review.  The  objective  of  the  review  was  to 
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determine  percentage  negotiated  cost  reductions  using  con- 
ventional cost  analysis  (non-Should-Cost)  techniques  which 
in  turn  would  be  used  as  the  basis  of  comparison  for  nego- 
tiated cost  reductions  using  the  Should  Cost  method. 

However,  the  researchers  were  unable  to  obtain  the  required 
cost  data  in  the  time  frame  allotted  for  the  preliminary 
investigation.  Therefore,  rather  than  pursuing  a review  of 
the  data  to  ascertain  percentage  negotiated  cost  reductions, 
the  investigation  turned  to  the  review  of  completed  con- 
tract cost  results  for  seven  of  those  contracts  negotiated 
using  the  Should  Cost  method.  The  reason  for  this  review 
was  to  assess  the  status  of  completed  contract  costs  as  a 
means  for  identifying  and/or  selecting  the  data  to  be  used 
in  the  research  study  to  be  performed  later. 

To  set  the  stage  for  the  later  research  work,  the 
preliminary  investigation  was  concerned  with  not  only  the 
availability,  by  contract,  of  total  cost  data,  but  also 
the  availability  of  sub-element  cost  data  in  the  format 
shown  on  the  DD  Form  633,  "Contract  Pricing  Proposal," 
Appendix  A.  By  a review  of  fou\r  Price  Negotiation  Memo- 
randa, an  administrative  document  for  recording  the 
results  of  contract  negotiations,  it  was  learned  that  cost 
data  by  sub-element  as  presented  on  the  DD  Form  633,  i.e.. 
Direct  Material,  Direct  Labor,  and  Indirect  Costs  generally 
were  neither  readily  available  nor  readily  reconcilable  to 
the  format  essential  to  the  contemplated  research  approach. 
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The  researchers  attribute  this  condition  to  the  fact  that 
the  contract  work  was  not  yet  completed  on  three  of  these 
four  contracts.  Therefore,  inasmuch  as  interim  cost 
progress  reporting  was  not  available  at  the  sub-element 
cost  level  of  detail,  the  contemplated  research  comparisons 
of  sub-element  costs  between  contracts  negotiated  using 
conventional  cost  analysis  and  those  using  Should  Cost 
could  not  be  made.  On  the  fourth  contract,  all  contract 
work  had  been  completed  and  the  final  contract  price  had 
been  settled  in  accordance  with  the  incentive  provisions  of 
the  Fixed-Price-Incentive  (FPI)  type  contract.  However, 
the  total  price  was  negotiated  and  no  separate  agreement 
was  made  with  the  contractor  regarding  the  sub-elements  of 
cost.  This  total  price  settlement  approach  was  supported 
by  a sub-element  cost  breakdown  of  what  the  Government 
negotiator  considered  to  be  the  individual  sub-element  cost 
settlement  and  could  have  been  used  to  support  the  origi- 
nally contemplated  research  approach.  The  one  contract 
represented  only  one  of  four  potential  items  to  be 
included  in  the  research  sample.  Therefore,  it  was  decided 
not  to  perform  future  research  at  the  sub-element  level  of 
cost  analyses  but  to  limit  the  analysis  associated  with 
testing  the  research  hypothesis  at  the  total  cost  level. 

The  preliminary  investigation  provided  further 
insight  into  cho  data  base,  specifically  with  respect  to 
the  accounting  for  the  effect  of  contract  changes  upon  the 


i 
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final  contract  cost.  A contract  cost  change  (modification) 
is  a legal  and  binding  change  to  the  contract  with  an 

attendant  increase  or  decrease  in  cost  as  the  considera-  ' 

tion  for  the  action.  More  specifically,  a contract  modifi- 
cation means  any  written  alteration  in  the  specification, 

delivery  point,  rate  of  delivery,  contract  period,  price,  ( 

\ 

quantity,  or  other  contract  provisions  of  an  existing  con-  j 

tract,  whether  performed  by  unilateral  action  in  accordance  \ 

j 

with  a contract  provision,  or  by  mutual  action  of  the  i 

parties  to  the  contract.  It  includes  bilateral  actions 
such  as  supplemental  agreements,  unilateral  actions  such 

j 

as  change  orders,  administrative  changes,  notices  of 

termination,  and  notices  of  the  exercise  of  a contract 

option  (21:1:15).  Peck  and  Scherer  have  stated  that  in  ( 

"3 

most  market  environments  a pervasive  optimise ic  bias  in  i 

sellers'  promises,  and  especially  in  price  quotations, 

would  have  dire  consequences — either  widespread  bankruptcy  | 

or  the  breakdown  of  traditional  contract  law.  They 
further  stated  that  in  the  nonmarket  environment  of 
advanced  weapons  acquisition,  competitive  optimism  flour- 
ishes since  the  penalties  for  underestimation  of  costs 
are  seldom  seve/e  (13:414).  Almost  every  weapons  systems 
acquisition  is  covered  by  some  sort  of  cost  reimbursable 
or  cost  redeterminable  provision  which  assures  a contrac- 
tor of  recovering  his  cost,  oftentimes  even  if  the  costs 
turn  out  to  be  significantly  higher  than  the  contractor's 
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original  estimates  (13:416).  This  situation  as  highlighted 
by  Peck  and  Scherer  Wc  s observed  during  the  preliminary 
investigation  performed  during  this  research  effort. 

An  example  of  this  type  situation  is  shown  in  Table  3. 


Table  3 

EXAMPLE  OF  EFFECT  OF  CONTRACT  CHANGES  ON 
NEGOTIATED  AND  FINAL  NEGOTIATED  CONTRACT  COST 


Contractor  #5 

Initial  Negotiated  Contract  Cost 

$112 

Million 

Add:  Contract  Changes  (41  Contract 

Actions) 

31 

Million 

Final  Negotiated  Contract  Cost 

$143 

Million 

Referring  to  the  above  table,  the  forty-one  contract 
changes  can  be  classified  into  three  categories: 

(1)  configuration  changes;  (2)  task  changes,  and  (3;  pro- 
gram changes.  These  categories  of  changes  have  been 
explained  as  follows  (5:363): 

Configuration  changes  alter  the  configuration  of  a 
system  being  built  for  delivery  to  the  Government. 
They  may  change  or  delete  an  existing  part,  or  add  a 
new  one.  For  example,  a change  that  alters  the 
structure  of  any  part  or  component  of  a production 
aircraft  is  a configuration  change.  Configuration 
changes  are  authorized  in  Engineering  Change  Pro- 
posals and  are  frequently  referred  to  as  engineering 
changes. 

A tark  change  does  not  usually  refer  to  changes  in 
hardware.  Task  changes,  for  example,  may  restruc- 
ture test  programs  or  feasibility  studies.  Although 
task  changes  are  not  like  engineering  changes,  in 
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practice  proposed  task  changes  are  usually  supported 
by  Engineering  Change  Proposals.  For  this  reason, 
task  changes  are  also  frequently  referred  to  as 
engineering  changes. 

Program  changes  involve  major— and  usually  very 
costly — revisions  to  quantities,  technical  perform- 
ance specifications,  delivery  schedules,  or  rate 
of  funding  for  a program. 

Every  negotiation  for  a change  order  occurs  in  a sole- 
source  environment  (5:376).  In  such  a case,  the  contrac- 
tor is  in  a strong  position  to  negotiate,  and  if  for 
example,  the  contractor  had  submitted  an  unrealistically 
low  proposal  (buy-in) , he  may  try  to  recoup  some  of  his 
losses  by  generating  numerous  contract  changes.  In  addi- 
tion, since  contractor  profits  normally  are  negotiated  as 
a percentage  of  cost,  the  contractor  may  be  motivated  to 
create  additional  cost  producing  tasks  as  a means  for 
acquiring  more  profit  (5:377).  Based  upon  the  Table  3 
(page  29)  observation  and  the  above  discussion,  the 

researchers  included  the  impact  of  contract  change  orders 

/' 

(revisions)  on  cost  and  profit  as  a variable  to  be  tested 
by  this  research  study. 

In  summary,  the  preliminary  investigation  was  a 
worthwhile  effort  which  facilitated  the  ease  of  performing 
the  later  research  work  and  indicated  that  it  was  a 
feasible  research  approach  to  compare  the  final  cost 
results  of  contracts  negotiated  using  conventional  cost 
analysis  techniques  tc  those  negotiated  using  the  Should 
Cost  technique. 
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Universe 

In  research  work,  the  entire  set  of  elements  to 
which  the  researcher  wishes  to  generalize  conclusions  is 
referred  to  as  the  universe.  The  universe  considered  in 
this  study  consisted  of  approximately  7,000  contracts 
negotiated  by  ASD  during  the  calendar  year  period  1971- 
1974.  However,  the  universe  contained  a variety  of  types 
of  contracts  of  which  all  but  twenty-three  were  negotiated 
using  conventional  cost  analysis  techniques.  Therefore, 
the  universe  had  little  or  no  bearing  on  the  sample  selec- 
tion used  in  this  research  and  is  mentioned  only  to  show 
the  relative  number  of  contracts  negotiated  using  the 
Should  Cost  method. 


Population  of  Interest 

Population  definition.  This  study  deals  with  a finite 
population;  that  is,  one  with  specific  limits  or  boundaries. 
While  the  universe  represents  the  entire  set  of  elements 
of  concern,  a population  is  defined  as  the  entire  set  of 
values  which  results  from  the  measurement  of  some  character- 
istics of  all  the  elements  of  the  universe.  This  measure- 
ment may  take  the  form  of  quantitative  variates  or  the 
presence  or  absence  of  some  qualitative  characteristic. 

As  previously  mentioned,  ASD  conducted  twenty- 
three  Should  Cost  analyses  during  the  time  period  under 
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study.  The  population  considered  in  this  research  study 
consisted  of  these  twenty-three  Should  Cost  analyses. 

Description.  In  performing  the  research  work,  the  popula- 
tion of  twenty-three  Should  Cost  studies  was  stratified 
into  two  subgroups:  one  subgroup  consisted  of  seventeen 

Fixed-Price- Incentive  (FPI)  contracts;  the  other  sub- 
group included  six  Firm  Fixed  Price  (FFP)  contracts.  FFP 
type  contracts  are  not  subject  to  a redetermination  of 
costs  after  the  contract  work  has  been  completed  (21:3- 
104);  therefore,  those  FFP  contracts  were  excluded  from 
the  population  of  interest  since  the  Government  does  not 
have  access  to  the  final  cost  data  as  a matter  of  routine 
cost  evaluation.  FPI  type  contracts  are  subject  to 
redetermination  of  costs  after  the  contract  work  has  been 
completed.  Under  FPI  type  contracts,  cost  savings  or 
overruns  are  shared  by  the  buyer  and  seller  in  accordance 
with  a predetermined  profit-sharing  formula.  With  most 
FPI  contracts,  the  contractor  is  responsible  for  all  costs 
in  excess  of  the  ceiling  price,  commonly  referred  to  as 
the  point-of-total  assumption,  or  the  point  where  the 
contract  effectively  reverts  to  a FFP  arrangement. 

The  remaining  seventeen  contracts,  negotiated 
using  the  Should  Cost  method,  comprised  the  stratified 
population  from  which  a convenience  sample  was  taken  for 
purposes  of  this  research  study. 
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Data  Collection 

To  test  the  research  hypothesis,  two  convenience 
samples,  one  consisting  of  conventional  cost  analysis 
and  the  other  consisting  of  Should  Cost  contracts,  were 
taken.  The  two  samples  were  categorized  for  testing  pur- 
poses as  Group  A (conventional  cost  analysis)  and  Group  B 
(Should  Cost) . 


Should  Cost  contracts.  During  the  initial  research,  it 
was  determined  that,  of  the  seventeen  Should  Cost  contracts 
in  the  stratified  population,  final  costs  were  available 
or  could  be  accurately  estimated  for  four  contracts. 

These  four  contracts  served  as  the  sample  of  convenience 
of  Should  Cost  contracts.  A convenience  sample  is  a sample 


that  is  obtained  when  practical  considerations*  force  the 


use  of  conveniently  available  data 
case) . 


(final  costs  in  thisis 


Conventional  cost  analysis  (non-Should  Cost)  contracts 


For  purposes  of  comparing  the  Should  Cost  to  non-Should 


Cost  contracts,  another  convenience  sample,  of 


conventional 


cost  analysis  contracts,  was  taken.  The  sample  consisted 

. . ■'! 

of  four  contracts;  one  each  representing  the  previous 
fiscal  year  buy  of  the  same  product  from  the  same  contrac- 
tor as  each  of  the  respective  four  Should  Cost  contracts. 
As  an  example,  a Should  Cost  study  was  performed  on  the 
FY72  procurement  from  Contractor  #3.  Therefore,  for 
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comparison  to  the  FY72  buy,  the  FY  1971  procurement  from 
Contractor  #3,  which  was  not  negotiated  under  Should  Cost 
guidelines,  was  selected  for  review. 

The  selection  of  the  contracts  representing  pre- 
vious year  buys  of  the  same  product  from  the  same  contrac- 
tor, as  the  convenience  sample  of  non-Should  Cost  con- 
tracts, was  made  to  facilitate  the  paired  comparison  of 
the  cost  outcomes  between  Should  Cost  and  non-Should  Cost 
contracts. 

Performance  results.  The  performance  results;  i.e.,  the 
contract  cost  outcomes  associated  with  the  various  stages 
of  contract  activity,  were.  collected  for  each  of  the 
contracts  in  the  two  samples.  This  cost  information  was 
obtained  by  examining  results  of  individually  documented 
Should  Cost  studies,  data  from  actual  contracts,  and  data 
from  contract  reports. 

The  results  of  the  individual  ShouLd  Cost  studies 
are  documented  in  Price  Negotiation  Memoranda  (PNM)  as 
no  separate,  formal  Should  .Cost  reports  are  issued  (22: 
7-1).  Similarly,  the  results  of  conventionally  negotiated 
contracts  are  included  in  the  PNM.  The  PNM,  which  serves 
as  the  record  of  negotiations,  and  is  completed  after  a 
contract^is^  ho<^bt4.4H\ed ,— contains  data  related  to  the  con- 
tractor's originally  proposed  cost/price,  the  Government's 
negotiation  objective,  and  the  negotiated  cost/price. 
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The  cost  information  collected  was  classified  into 
total  costs  for  each  of  the  five  stages  of  contract 
activity.  These  stages  of  contract  activity  are  listed 
and  defined  below. 

Proposed:  This  contract  activity  stage  represents 

the  contractor  proposal  action  taken  in  response  to 
the  Government's  initial  "Request  for  Proposal." 

The  total  cost  value  associated  with  this  stage  of 
contract  activity  represents  the  contractor's  initial 
contract  coat  proposal  in  millions  of  dollars. 

Negotiated:  This  contract  activity  stage  represents 

the  results  of  the  negotiation  action  relative  to 
establishing  the  contract  cost  (exclusive  of  profits 
or  fee) . The  total  cost  value  associated  with  this 
stage  of  contract  activity  represents  the  negotiated 
cost  proposal  in  millions  of  dollars. 

Revisions:  This  contract  activity  stage  represents 

the  contract  revisions  which  were  negotiated  as  a 
result  of  contract  change  activity  (see  Chapter  3, 
Preliminary  Investigation,  for  further  explanation 
of  contract  changes) . The  total  cost  value  associa- 
ted with  this  stage  of  contract  activity  represents 
the  negotiated  contract  cost  amount  in  millions  of 
dollars  for  the  several  changes  negotiated. 

Revised  Negotiated:  This  contract  activity  stage 

represents  the  results  of  the  negotiation  action 
relative  to  establishing  the  revised  contract  cost 
(exclusive  of  profit  or  fees) . The  total  cost  value 
associated  with  this  stage  of  contract  activity 
represents  the  total  revised  contract  ansi,  baseline 
in  millions  of  dollars  against  which  final  completed 
costs  are  compared. 

Completed;  This  contract  activity  stage  represents 
the  accumulation  process  of  gathering  all  costs  which 
have  been  incurred  in  the  performance  of  the  con- 
tract work.  The  total  cost  value  associated  with 
this  stage  of  contract  activity  represents  the  final 
actual  contract  costs  in  millions  of  dollars. 

The  data  pertaining  to  the  total  cost  outcomes  of 
each  stage  of  contract  activity  for  each  contractor  in 
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the  two  sample  groups  served  as  the  basis  for  analysis 
and  statistical  testing  which  are  discussed  in  Chapter  4. 

The  information  presented  on  the  next  page 
summarizes  the  delimiting  process  which  resulted  in  the 
obtaining  of  the  cost  data  used  in  this  research  (Figure 
1). 

Summary  List  of  Assumptions/ 

Limitations 

Assumptions.  The  assumptions  made  in  connection  with  this 
study  follow: 

1.  That  the  population  of  Should  Cost  studies 
were  conducted  in  accordance  with  AF  Pamphlet  70-5, 

"Should  Cost,"  which  sets  forth  the  Air  Force  concept  of 
and  approach  to  Should  Cost  as  well  as  establishes  detailed 
procedures  which  may  be  useful  in  conducting  the  Should 
Cost  reviews  (20: i).  As  the  same  criteria  were  applied 

to  the  conduct  of  these  Should  Cost  studies,  it  was 
assumed  that  the  sample  of  convenience  is  representative 
of  the  population  of  Should  Cost  contracts. 

2.  All  contract  work  had  not  been  completed  on 
three  of  the  four  Should  Cost  contracts  included  in  the 
sample  of  convenience.  For  those  three  contracts,  final 
costs  were  estimated  based  on  cost  data  and  percentage  of 
work  completed  (97  percent)  obtained  from  Cost  Performance 
Reports  or  Alternate  Management  Summary  Reports.  It 
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UNIVERSE 

(Approximately  7000  contracts  (ASD) 


y 

POPULATION  OF  INTEREST 
23  Should  Cost  contracts 


y 

STRATIFIED  POPULATION 

17  FPI  contracts 
6 FFP  contracts  (excluded) 


y 

POPULATION 

17  FPI  Should  Cost  contracts 


y 

SAMPLE 

4 contracts  (convenience  sample) 


y 

DATA  ELEMENTS 

Total  cost  outcomes  for  five  stages  of  contract  activity 


Figure  1 

Summary  of  Delimiting  Process 
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was  assumed  that  both  the  percentage  of  work  completed 
and  the  contract  costs  reported  were  accurate. 

Limitations.  The  limitations  of  this  study  are  as  follows t 

1.  The  study  was  not  intended  to  allow  for 
generalization  of  results  to  a larger  population  than  that 
surveyed  within  the  Aeronautical  Systems  Division. 

2.  The  Should  Cost  studies  conducted  during  the 
1971-1972  time  period  placed  less  emphasis  on  the  identi- 
fication of  long-term  contractor  management  recommenda- 
tions. Due  to  greater  concern  by  both  the  Air  Force  and 
the  General  Accounting  Office,  the  more  recent  (1973-1974) 
Should  Cost  studies  placed  greater  emphasis  on  identifica- 
tion and  implementation  of  long-term  contractor  management 
recommendations.  However,  these  management  type  recom- 
mendations generally  do  not  lend  themselves  to  quantita- 
tive assessment  due  to  their  long-term  nature;  an  inability 
for  existing  contractor  accounting  systems  to  isolate  the 
applicable  costs;  and  the  absence  of  appropriate  cost 
reporting  systems. 

3.  The  number  of  Should  Cost  contracts  where  the 
completed  actual  cost  data  were  available  was  limited. 

Final  costs  were  available,  or  could  be  accurately  esti- 
mated, for  four  contracts  from  the  population  of  seventeen 
Should  Cost  contracts. 

4.  The  availability  of  the  final  cost  data,  ns 
a practical  consideration,  forced  the  selection  of  the 
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four  contracts  as  a sample  of  convenience  in  lieu  of  a 
randomly  selected  sample  from  the  Should  Cost  contract 
population. 


5.  The  study  was  concerned  with  the  Air  Force 
conception  and  application  of  the  Should  Cost  technique 
vis-a-vis  that  of  the  Army,  Navy,  or  General  Accounting 
Office. 


6.  Table  4,  Group  A,  Contractor  #3  and  Contractor 
#4  (shown  in  Chapter  4)  include  data  that  were  adjusted 
by  the  researchers  to  reflect  a proposed  contract  cost 
amount  based  upon  a straight-line  extrapolation  of  3.9 
percent  upward  from  the  negotiated  amount  of  $166.2 
million. 


Chapter  4 
DATA  ANALYSIS 


General 

The  basic  data  used  in  this  research  was  obtained 
from  records  and  reports  maintained  by  the  Aeronautical 
Systems  Division  (ASD) . A general  description  of  the 
data  and  the  time  period  from  which  the  data  was  gathered 
are  explained  in  the  Population  of  Interest  section  of 
Chapter  3.  Details  of  the  data  collection  procedures 
will  be  explained  below. 

Data  collection  and  conversion.  As  explained  in  Chapter  3, 
the  two  samples  of  convenience  were  taken  and  categorized 
ass  Group  A — Contracts  Negotiated  Using  Conventional  Cost 
Analysis;  and  Group  B — Contracts  Negotiated  Using  Should 
Cost.  For  each  contract  in  these  two  sample  groups,  total 
cost  data  by  stage  of  contract  activity  (proposed, 
negotiated,  revisions,  revised  negotiated,  and  completed) , 
were  collected.  Table  4 depicts  these  cost  totals  for 
the  sample  contracts.  As  shown  in  the  table,  the  total 
cost  for  contractors  #3  and  #4  in  Group  A are  the  same. 

As  stated  above,  two  samples  of  convenience,  each  con- 
sisting of  four  contracts,  were  taken  as  a means  for  com- 
paring the  cost  results  of  contracts  negotiated  using 
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Table  4 


TOTAL  COSTS  PER  CONTRACT  ACTIVITY 


STAGE  OF  SAMPLE  CONTRACTS 


CONVENTIONAL  COST  ANALYSIS  (GROUP  A) 
(Total  Costs  $ Millions) 


Stages  of 

Contract 

Activity 

Contractor 

#1 

Contractor 

*2 

Contractor 

#3 

Contractor 

#4 

Proposed 

17.7 

2.7 

199.7/172.9** 

199.7/172.9** 

Negotiated 

15.7 

2.6 

192.0/166.2* 

192.0/166.2* 

Revisions 

1.1 

1.0 

7.2 

7.2 

Revised 

Negotiated 

16.8 

3.6 

173.4 

173.4 

Completed 

21.3 

4.9 

167.5 

167.5 

SHOULD  COST  (GROUP  B) 
(Total  Costs  $ Millions) 


Stages  of 


Contract 

Activity 

Contractor 

#1 

Contractor 

#2 

Contractor 

#3 

Contractor 

#4 

Proposed 

5.2 

2.0 

338.9 

227.9 

Negotiated 

4.9 

1.8 

266.2 

182.1 

Revisions 

-0- 

.2 

39.8 

24.6 

Revised 

Negotiated 

4.9 

2.0 

306.0 

206.7 

Completed 

4.9 

2.0 

259.5 

176.3 

♦Quantity  bought  was  reduced  due  to  lack  of  funds  thus 
resulting  in  a negotiated  base  amount  of  $166.2  million  versus  $192.0 
million  for  a higher  quantity. 

**To  facilitate  computations,  an  estimate  of  (172.9)  for 
the  proposed  amount  for  the  reduced  quantity  was  made  based  on  a 
straight-line  extrapolation  of  3.9  percent  upward  from  the  negotiated 
amount  of  166.2  of  the  reduced  quantity. 

Source:  ASD  Records  and  Reports. 
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conventional  cost  analysis  (Group  A)  versus  contracts 
negotiated  using  Should  Cost  (Group  B) . In  order  to  make 
the  paired  comparisons  between  the  two  samples,  by  con- 
tractor, a baseline  was  established.  In  these  cases,  the 
baseline  was  the  previous  year  or  last  contract  negotiated 
usin'j  the  conventional  cost  analysis  technique  with  the 
contractor  for  the  same  product.  Accordingly,  the  coded 
Contractor  #3  and  Contractor  it  4 cost  data  in  Table  4 were 
from  the  same  contract  since  there  were  two  consecutive 
Should  Cost  studies  performed  on  the  contractor  ror  two 
succeeding  product  buys.  This  data  generally  was  not 
readily  available  from  one  location;  therefore,  several 
visits  to  various  offices  were  required  to  obtain  the 
data.  The  following  represents  the  locations  and  source 
documentation  from  which  the  research  data  related  to  this 
study  were  obtained: 

Location 

Pricing  Directorate,  ASD 

System  Program  Office,  ASD 
Program  Control  Office 
Procuring  Contracting  Office 

Contract  Files  Control  Office,  ASD 

Procurement  Operations/Reports  Control  Office, 
ASD 

Documentation 

Price  Negotiation  Memoranda 

Cost  Performance  Reports 
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Alternate  Management  Summary  Reports 

Procuring  Contracting  Officer  Contract 
Document  Log 

The  cost  data  collected  by  contract  activity  stage 
was  then  converted  into  two  data  element  forms  suitable 
for  hypothesis  testing.  The  first  form  was  the  percent 
change  (increase/decrease)  of  cost  outcomes  between 
activity  stages.  In  this  case,  the  percent  changes  from: 

(1)  negotiated  to  proposed,  (2)  completed  to  proposed, 
and  (3)  completed  to  revised  negotiated  costs  were  com- 
puted. These  percent  changes  were  chosen  for  the  hypoth- 
esis testing  as  they  were  considered  by  the  researchers 
to  best  represent  measures  of  the  quantitative  cost  reduc- 
tion accomplishments  of  the  Should  Cost  efforts  to  date. 

The  other  form  was  the  ratio  or  percentage  of  the 
costs  of  contract  revisions  to:  (1)  the  cost  outcome  of 

the  proposed  stage  of  contract  activity,  and  (2)  the  cost 
outcome  of  the  negotiated  stage  of  contract  activity. 

These  percentages  were  chosen  for  the  hypothesis  testing 
because,  as  stated  in  the  preliminary  investigation  in 
Chapter  3,  the  percentages  reflect  the  effect  contract 
changes  (revisions)  have  upon  the  final  contract  cost 
outcome. 

The  cost  data,  having  been  converted  into  one  of 
the  two  forms  described  above,  was  then  used  to  make  the 
necessary  statistical  analyses  incidental  to  the  hypothesis 
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testing.  The  information  relating  to  the  converted  data 
for  each  contract  in  the  sample  groups  is  contained  in 
Appendix  B. 


Problems  in  data  collection.  As  previously  mentioned,  the 
primary  objective  of  this  research  study  was  to  quantify 
the  comparison  of  contract  cost  outcomes  on  Should  Cost 
negotiated  contracts  versus  those  negotiated  using  con- 
ventional cost  analysis  techniques.  To  accomplish  that 
objective,  it  ;as  essential  to  obtain  not  only  the 
initially  negotiated  contract  cost  data  but  completed 
actual  contract  cost  data  as  well.  Here  the  researchers 
experienced  two  basic  difficulties.  First,  there  was  a 
limited  number  of  Should  Cost  contracts  where  the  com- 
pleted actual  contract  cost  data  were  available.  This 
difficulty,  while  a limitation  for  this  research  study, 
could  be  overcome  through  replication  of  this  data  and  use 
of  an  expanded  sample  by  future  researchers.  The  next 
problem  associated  with  data  collection  was  that  of 
identifying  a single  repository  where  the  data  could  be 
located.  Earlier  in  this  chapter,  four  separate  locations/ 
offices  were  mentioned  where  the  required  data  were  found. 
During  the  research,  each  of  those  offices  was  visited  to 
obtain  the  required  data.  Although  each  of  the  offices 
supplied  valuable  data  to  the  overall  effort,  it  was  found 
that  the  System  Program  Office  was  often  the  loca- 
tion with  the  most  usable  repository  of  data. 
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Hypothesis  Testing 

It  was  determined  by  the  researchers  that  hypo- 
thesis testing  performed  on  three  combinations  of  paired 
data  elements  from  the  sample  groups  would  be  appropriate 
in  the  testing  of  the  basic  research  hypothesis.  Using 
the  calculated  data  previously  described,  the  average 
differences  between  the  two  sample  groups  of  the  percent 
changes  from:  (1)  Proposed  to  Negotiated,  (2)  Proposed  to 

Completed,  and  (3)  Revised  Negotiated  to  Completed  costs 
were  compared  and  tested  for  statistical  significance  by 
construction  of  a confidence  interval  for  A around  each 
average  difference.  A description  of  the  results  of  the 
testing  of  each  data  combination  follows. 

Combination  1 — Proposed  Cost  to  Negotiated  Cost.  For  this 
test,  the  percentage  changes  of  the  Proposed  to  Negotiated 
costs  for  each  of  the  four  sample  contracts  negotiated 
using  conventional  cost  analysis  and  negotiated  using  the 
Should  Cost  method  were  compared.  Next,  the  difference 
between  the  percent  change  in  each  paired  group  (non- 
Should  Cost  versus  Should  Cost)  was  computed  (see  Table  5) 
and  an  average  difference  (5)  of  8.65  percent  for  the  two 
groups  was  established.  A standard  deviation  (SD)  of  10.7 
percent  was  computed  using  the  5.  A ninety-five  percent 
confidence  interval  for  A,  around  the  5,  was  calculated 


as  follows: 


C.I.  = 8.65  + 3.18  (^7-^ 

C.I.  = -8.36  to  25.66 

Below  is  the  graphical  representation  of  the  confidence 
interval. 


Average  Difference  (A) 
in  the  Population 


-8.36 


Confidence  Interval 


Since  the  null  hypothesis  hypothesized  that  A equals  zero 
and  since  the  confidence  interval  for  A encompasses  zero, 
the  null  hypothesis  cannot  be  rejected. 


Combination  2 — Proposed  Cost  to  Completed  Cost.  In  this 
test,  the  percent  change  of  the  Proposed  to  Completed 
costs  for  each  of  the  four  sample  contracts  negotiated 
using  conventional  cost  analysis  and  negotiated  using  the 
Should  Cost  method  were  compared.  The  difference  between 
the  percent  change  in  each  paired  group  (non-Should  Cost 
versus  Should  Cost)  was  then  computed  (see  Table  6)  and 
an  average  difference  (5)  of  36.85  percent  for  the  two 
groups  was  established.  A standard  deviation  (SD)  of 
29.9  percent  was  computed.  A ninety-five  percent  con- 
fidence interval  for  A around  the  5 was  calculated  as 


follows: 


^55^5^'*^' v 'W^ffl^^^g^^r^wyig^K^  "vwwgr  *: 
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C.I.  =-10.69  to  84.4 

Below  is  the  graphical  representation  of  this  confidence 
interval. 


Average  Difference  (A) 
in  the  Population 


-10.69  0 

— < h 


84.4 
— 4- 


Confidence  Interval 


Since  the  null  hypothesis  hypothesized  that  A equals  zero 
and  since  the  confidence  interval  for  A encompasses  zero, 
the  null  hypothesis  cannot  be  rejected. 


Combination  3 — Revised  Negotiated  Cost  to  Completed  Cost. 
For  this  test,  the  percent  change  of  the  Revised  Negotiated 
to  Completed  costs  for  each  of  the  sample  contracts  nego- 
tiated using  conventional  cost  analysis  and  negotiated 
using  the  Should  Cost  method  were  compared.  The  differ- 
ence between  the  percent  change  in  each  paired  group  (non- 
Should  Cost  versus  Should  Cost)  was  then  computed  (see 
Table  7)  and  an  average  difference  (5)  of  21.85  percent 
for  the  two  groups  was  established.  A standard  deviation 
(SD)  of  14.64  percent  was  computed.  A ninety-five  percent 
confidence  interval  for  A around  6 was  calculated  as 


follows: 
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C.I.  - D 


1 1-025  (yr) 

C.I.  = 21.85  + 3.18  ("—%--■) 


C.I.  = -1.43  to  45.13 

Below  is  the  graphical  representation  of  this  confidence 
interval . 


Average  Difference  (A)  -1.43  0 45.13 

in  the  Population  . . . 


Confidence  Interval 


Since  the  null  hypothesis  hypothesized  that  A equals  zero 
and  since  the  confidence  interval  for  A encompasses  zero, 
the  null  hypothesis  cannot  be  rejected. 

In  addition  to  the  hypothesis  testing  using  the 
three  cost  combinations  cited  above,  statistical  testing 
of  the  significance  of  the  difference  between  the  two 
paired  sample  groups  {non-Should  Cost  versus  Should  Cost) 
of  the  costs  of  contract  Revisions  as  a percentage  of  both 
the  Proposed  cost  and  the  initial  Negotiated  cost  was 
conducted.  The  results  of  the  two  additional  tests  are 
described  below. 


Costs  of  Revisions  as  percentage  of  Proposed  cost.  In 
this  test,  the  values  for  the  costs  of  Revisions  as  a 
percentage  of  the  Proposed  costs,  for  each  of  the  four 
sample  contracts  negotiated  using  conventional  cost 


52 

analysis  and  negotiated  using  the  Should  Cost  method,  were 
compared.  Then,  the  difference  between  the  percentages 
in  each  paired  group  (non-Should  Cost  versus  Should  Cost) 
was  computed  (see  Table  8)  and  an  average  difference  (5) 
of  4.78  percent  for  the  two  groups  was  established.  A 
standard  deviation  (Sg)  of  16.08  percent  was  computed 
using  the  5.  A ninety-five  percent  confidence  interval 
for  A,  around  the  D,  was  calculated  as  follows: 

c-I-a5±t-o25  fe) 

C.I.  = 4.78  + 3.18  ) 

C.I.  * -20.8  to  30.3 

The  graph  for  this  confidence  interval  is  below. 

Average  Difference  (A)  -20.8  0 30.3 

in  the  Population ^ i ) _ 

v V ' 

Confidence  Interval 

Since  the  null  hypothesis  hypothesized  that  A equals  zero 
and  since  the  confidence  interval  for  A includes  zero,  the 
null  hypothesis  cannot  be  rejected. 

Costs  of  Revisions  as  percentage  of  Negotiated  costs.  For 
this  test,  the  values  for  the  costs  of  Revisions  as  a per- 
centage of  the  Negotiated  costs,  for  each  of  the  four  sample 
contracts  negotiated  using  conventional  cost  analysis  and 
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negotiated  using  the  Should  Cost  method,  were  compared. 

The  difference  between  the  percentages  in  each  paired 
group  (non-Should  Cost  versus  Should  Cost)  was  computed 
(see  Table  9)  and  an  average  difference  (6)  of  .53  percent 
for  the  two  groups  was  established.  A standard  deviation 
(SD)  of  12.52  percent  was  computed  using  the  5.  A ninety- 
five  percent  confidence  interval  for  A,  around  5,  was 
calculated  as  follows: 

C.I.  = 5 + t 

C.I.  = .53  + 3.18 
C.I.  *-19.38  to  20.44 


The  graphical  representation  for  this  confidence  interval 
is  depicted  below. 


Average  Difference  (A) 
in  Population 


-19.38 
( 


20.44 




Confidence  Interval 


As  the  null  hypothesis  hypothesized  that  A equals  zero 
and  since  the  confidence  interval  for  A encompasses  zero, 
the  null  hypothesis  cannot  be  rejected. 


Chapter  5 

CONCLUSIONS  AND  RECOMMENDAT IONS 

This  chapter  contains  the  conclusions  and  associ- 
ated recommendations  developed  as  a result  of  this  research 
study.  In  addition,  recommendations  for  future  research  on 
the  Should  Cost  subject  are  also  presented. 

Conclusions 

Resulting  from  statistical  analysis  findings.  The  research 
hypothesis  tested  was; 

There  is  a difference  between  the  actual  cost  out- 
comes on  contracts  negotiated  using  Should  Cost  methods 
and  the  actual  cost  outcomes  on  those  contracts  negotiated 
using  conventional  cost  analysis  techniques. 

The  statistical  test  results  did  not  support  this 
hypothesis.  As  indicated  previously  in  Chapter  3,  sta- 
tistical hypotheses  were  formulated  to  test  the  research 
hypothesis.  Classical  hypothesis  testing  methods  were 
used  and  the  research  hypothesis  or  assumed  condition 
became  the  "alternate"  hypothesis  for  test  purposes.  The 
basic  hypothesis  that  there  is  no  difference  between  the 
cost  outcomes  of  the  two  contract  populations  became  the 
"null"  hypothesis.  For  the  research  hypothesis  (alternate 
hypothesis)  tc  be  supported,  the  statistical  testing  must 
result  in  rejection  of  the  null  hypothesis. 
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Five  separate  statistical  tests  were  conducted 
to  test  the  research  hypothesis.  The  findings  resulting 
from  each  of  the  statistical  tests  are  described  in 
Chapter  4.  These  findings  as  they  relate  to  the  conclu- 
sions concerning  the  research  hypothesis  are  discussed 
below. 


Test  #1 — Proposed  cost  to  Negotiated  cost.  The 
finding  resulting  from  this  statistical  test  indicated 
that  the  null  hypothesis  cannot  be  rejected.  Therefore, 
it  can  be  concluded  from  the  data  there  is  no  difference, 
at  the  .05  significance  level,  between  the  percent  change 
from  the  Proposed  to  Negotiated  costs  for  contracts  nego- 
tiated using  conventional  cost  analysis  and  for  those 
negotiated  using  the  Should  Cost  method. 

Test  #2 — Proposed  cost  to  Completed  cost.  The 
finding  resulting  from  this  statistical  test  indicated 
that  the  null  hypothesis  cannot  be  rejected.  Therefore, 
it  can  be  concluded  from  the  data  there  is  no  difference, 
at  the  .05  significance  level,  between  the  percent  change 
from  the  Proposed  to  Completed  costs  for  contracts  nego- 
tiated using  conventional  cost  analysis  and  for  those 
negotiated  using  the  Should  Cost  method. 

Test  #3— Revised  Negotiated  cost  to  Completed  cost. 
The  finding  resulting  from  this  statistical  test  indicated 
that  the  null  hypothesis  cannot  be  rejected.  Therefore, 
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it  can  be  concluded  from  the  data  there  is  no  difference, 
at  the  .05  significance  level,  between  the  percent  change 
from  the  Revised  Negotiated  to  Completed  costs  for  con- 
tracts negotiated  using  conventional  cost  analysis  and 
for  those  negotiated  using  the  Should  Cost  method. 

Test  #4 — Costs  of  Revision  as  Percentage  of  Pro- 
posed cost.  The  finding  resulting  from  this  statistical 
test  indicated  that  the  null  hypothesis  cannot  be  rejected. 
Therefore,  it  can  be  concluded  from  the  data  there  is  no 
difference,  at  the  .05  significance  level,  between  the 
percentage  of  the  costs  of  Revisions  to  the  Proposed  costs 
for  contracts  negotiated  using  conventional  cost  analysis 
and  for  these  negotiated  using  the  Should  Cost  method. 

Test  #5 — Costs  of  Revisions  as  Percentage  of 
Negotiated  cost.  The  findings  resulting  from  this  sta- 
tistical test  indicated  that  the  null  hypothesis  cannot 
be  rejected.  Therefore,  i<-  can  be  concluded  from  the 
data  there  is  no  difference,  at  the  .05  significance  level, 
between  the  percentage  of  the  costs  of  Revisions  to  the 
Negotiated  costs  for  contracts  negotiated  using  conven- 
tional cost  analysis  and  for  those  negotiated  using  the 
Should  Cost  method. 

The  statistical  tests  measured  the  statistical  significance 
of  quantifiable  cost  outcomes  at  various  stages  of  contract 
activity  with  the  objective  of  supporting  the  research 


59 


hypothesis.  However,  the  results  of  the  testing  did 
not  support  the  research  hypothesis.  From  the  data  used 
in  this  research  study,  it  cannot  be  conclusively  stated 
that  there  is  a difference  between  the  actual  cost  out- 
comes on  contracts  negotiated  using  Should  Cost  methods 
and  the  actual  cost  outcomes  on  those  contracts  negoti- 
ated using  conventional  cost  analysis  techniques.  A 
summary  of  the  results  of  the  statistical  hypothesis 
testing  is  contained  in  Table  10. 

Resulting  from  corollary  findings.  As  discussed  earlier 
in  connection  with  the  preliminary  investigation  findings, 
contract  changes  (revisions)  represent  a significant  cost 
and  profit  impact  on  most  major  procurements.  During  the 
research,  it  was  found  that  significant  contract  price 
changes  did  develop  on  those  contracts  where  Should  Cost 
was  applied,  a situation  not  unlike  that  found  on  those 
contracts  which  were  negotiated  using  conventional  cost 
analysis  techniques.  A review  of  those  changes  did  show 
that  change  activity  increased  contract  target  costs  on 
three  of  the  four  contracts  included  in  the  Should  Cost 
convenience  sample.  Most  significant,  however,  as  shown 
on  Tables  3B  and  4B  (pages  81  and  82  respectively) , 
contract  changes  were  negotiated  which  added  both  cost 
and  related  profit  to  the  contracts.  Tables  3B  and  4B 
do  not  reflect  the  added  profit;  however,  the  profit  can 
be  estimated  by  applying  a 10.7  percent  factor  to  the 
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"revisions"  total  cost  amount.  The  10.7  percent  factor 
was  obtained  from  the  Price  Negotiation  Memoranda  which 
documented  the  results  of  the  respective  contract  negoti- 
ations. An  analysis  of  the  final  cost  outcome  for  two 
of  these  Group  B contracts  indicated  that  the  contractor 
in  both  instances  underran  the  initially  negotiated 
target  cost#  hence  a situation  where  the  change  activity 
cultivated  a potential  "windfall"  profit  situation.  For 
example,  the  Group  B contract  as  shown  on  Table  3B  (page 
81)  provided  for  a 10.7  percent  target  profit  factor  and 
a 50  percent/50  percent  under  target  share  ratio.  The 
"windfall"  profit  created  by  this  situation  (revisions) 
could  approximate  $19. 9 million  dollars,  calculated  as 
follows: 

$39.8  million  X 50%  = $19.9  million  . 

Summary.  In  summary,  it  is  of  importance  to  note  that 
the  sample  of  convenience  included  just  four  contracts 
out  of  a total  of  twenty-three  contracts  which  had  been 
negotiated  using  the  Should  Cost  method.  The  question 
then,  is  that  sample  large  enough  to  establish  statistical 
significance?  (Statistical  significance  simply  means 
that  enough  data  have  been  collected  to  establish  that  a 
difference  does  exist.  It  does  not  mean  that  the  differ- 
ence is  necessarily  important  (23:188)).  An  intuitive 
answer  in  this  case  is  that  probably  the  sample  is  not 
large  enough  to  conclusively  indicate  that  Should  Cost 
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does  produce  cost  outcomes  different  than  conventionally 
negotiated  contract  results. 

The  research  performed  during  this  study,  while 
based  on  a limited  sample,  does  give  support  to  the  belief 
that  Should  Cost  may  be  producing  results  less  than  those 
originally  anticipated  from  its  use.  At  the  same  time, 
a corollary  finding  indicated  that  the  use  of  the  tech- 
nique may  give  rise  to  greater  or  continued  use  of  contract 
changes  which  could  result  in  "windfall"  profits  notwith- 
standing the  fact  the  original  Should  Cost  study  was  effec- 
tive. Accordingly,  appropriate  recommendations  regarding 
these  issues  will  be  offered  as  a basis  for  further 
research  in  this  subject  area. 


Recommendations 


Based  upon  the  findings  and  conclusions  of  the 
research,  the  following  recommendations  are  made: 


A.  To  discourage  the  use  of  contract  changes  as 


a potential  strategy  usable  by  firms  who  have  negotiated 
tight  dontract  cost  targets  as  a result  of  Should  Cost, 
it  is  recommended  that:  (1)  the  Government  closely 


evaluate  the  need  for  the  proposed  change;  and  (2)  con- 

I 

sidcr  the  use  of  a "no  fee  or  profit"  policy  below  a 
certain  dollar  threshold  for  these  added  efforts. 

B.  A is  recommended  that  the  research  approach 
taken  by  this  study  be  used  to  undertake  an  expanded 
study  of  the  results  of  Should  Cost  activity  experienced 
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by  the  other  Military  Services  as  well  as  additional 
Should  Cost  activity  which  will  have  been  concluded  by 
the  Air  Force. 
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Chapter  6 
FUTURE  RESEARCH 

As  a result  of  the  research  work  performed  in  this 
effort,  certain  postulations  were  made  which  were  con- 
sidered significant  enough  to  warrant  further  mention  in 
this  thesis  report.  It  is  intended  that  these  postulates 
be  considered  as  a basis  for  further  research  which  could 
be  performed  as  a means  for  illuminating  the  deficiencies 
which  Should  Cost  seeks  to  correct. 


Evolutional  Pattern  and  Environ- 
ment of  Should  Cost 

Numerous  articles  and  research  works  have  been 
published  generally  dealing  with  the  "how  to"  and  "by 
whom"  mechanics  for  conducting  Should  Cost  reviews.  As 
previously  mentioned,  this  research  study  sought  to  com- 
pare the  effectiveness  of  final  cost  outcomes  for  those 
contracts  negotiated  using  Should  Cost  to  the  final  cost 
outcomes  for  those  contracts  negotiated  using  conventional 
cost  analysis.  While  the  results  of  this  research  did 
indicate  that  it  can  be  concluded  that  there  is  no  differ- 
ence, at  the  .05  significance  level,  between  those  actual 
cost  outcomes  for  Should  Cost  versus  non-Should  Cost 
negotiated  contracts,  it  remains  that  a more  in-depth 
analysis  of  Should  Cost  over  the  long-term  is  in  order. 
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The  degree  of  analysis  developed  in  this  chapter  will  be 
limited  to  a summary  treatment  of  a significant  issue  as 
a means  for  setting  the  stage  for  future  research;  namely, 
organizational  behavioral  implications  of  the  Should  Cost 
technique.  Should  Cost  as  applied  by  the  DOD  has  taken 
an  accounting  and  industrial  engineering  approach  toward 
a complex  problem.  Inherent  in  the  acquisition  of  major 
weapon  systems  is  the  constant  interaction  of  multi- 
disciplined  Government  and  contractor  organizations  such 
as  the  Defense  Contract  Administration  Services  (DCAS) , 
Defense  Contract  Audit  Agency  (DCAA) , and  major  commodity 
buying  actuvities  such  as  ASD  of  the  Air  Force  Systems 
Command  (AFSC).  Within  that  interactive  system  are 
processes  which  require  or  produce  cost  estimates,  over 
time,  related  to  the  acquisition  at  hand.  Those  activi- 
ties are  conducted  by  the  Government  and  its  contractors 
in  an  atmosphere  conducive  to  contract  negotiations. 
However,  Should  Cost  interjects  itself  at  a finite  point 
in  time  with  the  seemingly  insurmountable  objective  of 
predicting  what  a system,  subsystem,  or  component  ought 
to  cost,  given  reasonably  attainable  efficient  standards 
of  operation.  This  quantitatively-oriented  view  of  the 
problem  seems  to  discount  the  organizational  behavior 
relationships  essential  for  achieving  the  longer  range 
objectives  of  efficient  operation.  The  significance  of 
the  above  statement  is  that  Should  Cost  may  be  an  ad  hoc 
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answer  to  a deeply  rooted  organizational  behavior  problem 
such  as  a roles  and  missions  question  between  the  partici- 
pating organizations  which  could  result  in  duplications 
of  effort.  The  ad  hoc  nature  of  the  approach  may  sub- 
optimize to  achieve  near  term  economic  relief  in  the  form 
of  lowered  contract  price  objectives.  However,  the 
reality  of  this  ad  hoc  approach  is  that  it  manifests 
itself  as  a change  to  an  organizational  structure  without 
allowing  for  assessment  of  it3  impact  on  the  existing 
organizational  structure  represented  by  the  Defense  and 
industry  establishments. 

The  DOD  has  organized  functionally  to  conduct  its 
acquisition  activities.  This  organizational  structure 
has  produced  characteristics  expected  of  such  an  approach: 
specialization,  centralization,  systemization,  and  process 
configuration.  The  following  discussion  is  offered  as  a 
means  to  better  relate  these  characteristics  to  the  DOD 
organization  with  respect  to  the  acquisition  process.  DOD 
organizations  such  as  DCAS,  DCAA,  and  major  commodity 
buying  activities  such  as  ASD  are  representative  of  such 
specialized  organizations.  Basic  to  such  an  organizational 
structure  is  the  reduction  of  centralized  authority  with 
the  offsetting  phenomena  of  increased  authority  configura- 
tion or  chain  of  command.  The  activities  of  the  various 
organizations  are  so  diverse,  yet  virtually  essential  to 
the  overall  acquisition  process,  that  they  must  be 
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conducted  in  a systematized  fashion,  hence  the  need  for 
regulations,  rules  and  other  codes  of  operation.  The 
idea  of  process  configuration  is  synomymous  with  integra- 
tion or,  in  other  words,  once  work  is  broken  out,  it  must 
be  pulled  together  to  accomplish  the  overall  acquisition 
objective.  But  back  to  the  basic  question:  Where  does 

Should  Cost  fit  into  this  structure?  At  the  risk  of  over- 
simplification, it  can  be  said  that  Should  Cost  is  a means 
for  correcting  problems  caused  by  inadequacies  associated 
with  a functionally  organized  activity.  While  the  label 
"Should  Cost"  serves  the  valuable  purpose  of  broadcasting 
the  importance  of  improving  efficiency  and  reducing  cost, 
it  at  the  same  time  diverts  attention  from  the  vital  issue 
of  organizational  relationships  basic  to  the  survival  of 
any  institution.  This  can  best  be  characterized  by  William 
G.  Scott’s  comment  (14:665): 

Few  segments  of  society  have  engaged  in  organizing 
more  intensively  than  business.  The  reason  is  clear. 
Business  depends  on  what  organization  offers.  Business 
needs  a system  of  relationships  among  functions;  it 
needs  stability,  continuity,  and  predictability  in 
its  internal  activities  and  external  contacts. 

Business  also  appears  to  need  harmonious  relationships 
among  the  people  and  processes  which  make  it  up.  Put 
another  way,  a business  organization  has  to  be  free, 
relatively,  from  destructive  tendencies  which  may  be 
caused  by  divergent  interests. 

The  above  cursory  treatment  of  a complex  issue 
which  is  beyond  the  scope  of  this  thesis,  leaves  room  for 
greater  explanations  and  more  intense  investigation  which 
could  be  undertaken  as  a separate  research  effort.  However, 
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the  significant  point  that  can  be  made  was  made  by  Scott, 
and  that  essentially  is  that  business  needs  a system  of 
harmonious  relationships  among  functions,  along  with 
stability,  continuity  and  predictability.  Should  Cost 
as  an  ad  hoc  technique  violates  at  least  the  stability 
and  continuity  features  of  the  business  relationship. 
However,  the  results  of  this  "violation"  could  be  either 
favorable  or  unfavorable.  In  the  case  of  the  favorable 
outcome,  it  can  be  said  that  Should  Cost  provides  the 
motivation  to  purge  organizational  inertia  from  both  the 
contractor  organization  and  Government  contract  administra- 
tion organization  as  well.  In  the  case  of  the  unfavorable 
outcome,  the  Should  Cost  approach  can  disrupt  the 
stability  and  continuity  inherent  in  an  effective  con- 
tractor and  Government  contract  administration  activity. 
Therefore,  the  real  question  is:  Will  the  DOD-Industry 

relationship  thrive  or  suffer  under  the  super imposition 
of  this  ad  hoc  technique  over  a business-oriented  institu- 
tion? 

Production — An  Integral  Part 
of  the  Should  Cost  Technique 

The  economic  environment  associated  with  the  type 
of  acquisition  on  which  Should  Cost  is  used  is  one  where 
the  Government  is  a captive  customer,  subject  to  the  con- 
straints common  to  a situation  where  there  is  no  competi- 
tion (10:16-24).  Costs  are  functions  of  the  production 
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processes  inherent  in  the  activities  performed  by  any 
manufacturing  organization.  The  key  underlying  these 
activities  is  the  production  process  which  inevitably  is 
translated  into  a common  denominator,  namely  contract  cost 
or  price.  Gary  E.  Hagen  (7:1-15)  stated  that  the  term 
"Production"  is  subject  to  a variety  of  definitions.  As 
an  example,  the  Air  Force  Systems  Command  has  said  that 
production  is  difficult  to  define  so  that  it  means  the 
same  to  ACOs,  PCOs,  program  directors,  contractors,  and 
Air  Force  production  specialists  (1*:9).  This  result  is 
not  unlike  that  experienced  by  practitioners  and  recipi- 
ents of  the  Should  Cost  technique.  Therefore,  to  set  the 
stage  for  further  discussion  of  Should  Cost  from  a produc- 
tion viewpoint,  the  following  definition  of  production  is 
offered:  Production  includes  all  processes  and  procedures 

designed  to  tr?  sform  a set  of  input  elements  into  a 
specific  output  element.  Its  major  functions  include 
design  or  producibility,  production  planning,  production 
control,  production  demonstration  and  testing,  manufactur- 
ing method  development,  fabrication,  assembly,  installation, 
checkout,  scheduling,  and  production  program  surveillance 
(19:1-5).  This  definition  is  comprehensive  and  covers 
many  of  t^e  underlying  activities  embodied  in  Should  Cost. 
Mr.  Hagen's  discussion  of  production  management  as  it 
applies  during  system  acquisition  provided  insight  into 
the  basic  underlying  production  foundation  which  Should 
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Cost  seeks  to  treat  in  an  ad  hoc  fashion  (7).  He  raised 
the  issue  that  the  constant  change  in  manufacturing  tech- 
nology over  the  years  has  outstripped  the  knowledge  and 
skill  levels  of  many  of  the  Air  Force  personnel  assigned 
to  the  production  function  (19:3).  This  is  a void  which 
Should  Cost  has  sought  to  fill. 

The  1960s  was  a period  fraught  with  cost  growth 
and  fixed  price  type  contracting.  The  assumption  during 
that  era  essentially  was  that  fixed  price  incentive  con- 
tracting provided  sufficient  profit  motivation  and  risk 
assumption  by  contractors  to  promote  efficient  production 
of  defense  material  items.  The  resultant  contracting 
activity  led  to  a change  in  the  Government  role  from  one 
of  heavy  involvement  to  one  of  surveillance.  The  outcome 
of  that  era  as  now  being  experienced  during  the  1970s 
is  greater  use  of  cost  type  contracting,  hence  greater 
Government  engagement  with  contractors  in  the  management 
of  their  mutual  objectives. 

In  summary,  production  can  be  construed  to  be  the 
central  theme  of  Should  Cost.  However,  Should  Cost 
usually  occurs  at  a finite  point  in  time  and  on  an  ad  hoc 
be  ?is  during  the  production  phase  of  acquisition.  Produc- 
tion planning  and  operations,  on  the  other  hand,  are  con- 
stant processes  that  span  the  breadth  of  the  acquisition 
cycle.  This  point  is  supported  by  the  Air  Force  Systems 
Command  statement  that  the  success  of  the  production  phase 
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is  dependent  in  large  measure  on  the  extent  and  quality  of 
the  planning  and  assessment  of  work  accomplished  during 
the  development  cycle  of  the  acquisition  process  (18:16). 

It  is  essential  to  again  point  out  that  production 
oroces- es  and  planning  are  the  key  functions  which  need  to 
be  considered  by  the  Government  and  its  contractors  on  a 
continuing  day-to-day  basis  if  the  mutual  interests  of 
both  parties  and  the  nation  are  to  be  preserved. 

Summary  of  and  Recommendation 
for  Future  Research 

Should  cost  has  proved  to  be  a valuable  method  for 
calling  attention,  from  both  within  the  Government  and 
within  industry,  to  the  need  for  greater  contractor  plant- 
wide cost  effectiveness  (11:19).  However,  in  all  proba- 
bility, as  evidenced  by  the  Chapter  4 findings,  contrac- 
tors may  have  adjusted  to  Should  Cost  and  the  increased 
adversary  atmosphere  created  by  its  application.  To 
incur  lasting  effect  from  Should  Cost's  initial  impact, 
it  seems  that  both  Government  and  industry  together  must 
mutually  promote  production/industrial  engineering  related 
efficiencies  which  are  basic  to  the  Should  Cost  technique. 
This  undertaking  will  not  be  something  which  can  be  man- 
dated; rather,  it  must  be  initiated  in  a cooperative  vein 
with  participation  from  all  concerned  organizations.  These 
organizations  include  the  Service  buying  activities. 


Defense  Contract  Administration  Services  (DCAS) , Air  Force 
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Plant  Representative  Offices  (APPRO) , Defense  Contract 
Audit  Agency  (DCAA) , and  contractors. 

It  is  believed  that  a program  of  this  nature  could 
be  initiated  at  the  Office  of  Secretary  of  Defense  (OSD) 
level  in  the  form  of  a "Management  by  Objectives"  approach 
whereby  the  Office  of  the  Secretary  of  Defense,  the  Ser- 
vices, and  the  separate  Defense  Agencies  jointly  identify 
their  common  goals,  define  each  organization's  maior  areas 
of  responsibility  in  terms  of  the  results  expected  of  it, 
and  use  these  measures  as  guides  for  directing  the  indi- 
vidual organization's  day-to-day  efforts  towards  accomplish- 
ing the  objectives  sought  by  Should  Cost  on  an  ad  hoc  basis 
(6:245-251;  12:13). 

Based  upon  the  above  discussion,  it  is  recommended 
that  research  be  conducted  with  respect  to  the  impinging 
organizational  behavioral  impact  generated  as  a result  of 
using  the  ad  hoc  Should  Cost  technique. 


APPENDIX  B 

CONVERTED  CONTRACT  COST  DATA 
USED  IN  HYPOTHESIS  TESTING 

As  defined  in  Chapter  4,  the  cost  data  collected 
by  contract  activity  stage  for  each  sample  contract  was 
converted  into  one  of  two  data  element  forms  suitable  for 
hypothesis  testing.  The  two  forms  were:  (1)  percent 

change  (increase/decrease)  of  cost  outcomes  between  con- 
tract activity  stages  and  (2)  ratio  of  percentage  of  the 
costs  of  contract  revisions  to  proposed  costs  and  to 
negotiated  costs.  The  computations  performed  to  convert 
the  cost  data  into  testing  form  are  described  below. 


Percent  Change  (Increase/Decrease) 
of  Cost  Outcomes  Between  Contract 
Activity  Stages 


(1)  Negotiated  to  Proposed:  The  proposed  cost 

was  used  as  the  baseline  in  this  computation.  The  pro- 
posed cost  total  was  divided  into  the  difference  (increase/ 
decrease)  between  the  proposed  cost  and  the  negotiated 
cost.  The  resulting  decimal  was  then  multiplied  by  100 

to  convert  the  decimal  to  a percent. 

(2)  Completed  to  Proposed:  The  proposed  cost  was 

used  as  the  baseline  in  this  computation.  The  proposed 
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cost  total  was  divided  into  the  difference  (increase/ 
decrease)  between  the  proposed  cost  and  the  completed  cost. 
The  resulting  decimal  was  then  multiplied  by  100  to  con- 
vert the  decimal  to  a percent. 

(3)  Completed  to  Revised  Negotiated;  The  revised 
negotiated  cost  was  used  as  the  baseline  in  this  computa- 
tion. The  revised  negotiated  cost  total  was  divided  into 
the  difference  (increase/decrease)  between  the  revised 
negotiated  cost  and  the  completed  cost.  The  resulting 
decimal  was  multiplied  by  100  to  convert  the  decimal  to  a 
percent. 

Ratio  or  Percent  of  the  Costs  of 
Contract  Revisions  to  Proposed  and 
to  Negotiated  Costs 

(1)  Proposed : In  this  computation,  a ratio  was 

formed  by  dividing  the  proposed  cost  total  into  the  total 
cost  of  the  contract  revisions.  The  resulting  decimal 
was  multiplied  by  100  to  convert  the  decimal  to  a percent. 

(2)  Negotiated:  In  this  computation,  a ratio  was 

formed  by  dividing  the  negotiated  cost  total  into  the 
total  cost  of  the  contract  revisions.  The  resulting 
decimal  was  multiplied  by  100  to  convert  the  decimal  to 

a percent. 

The  converted  cost  data  computed  for  each  of  the 


1 
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sample  contracts  and  used  in  the  statistical  hypothesis 
testing  is  shown  in  Tables  IB  through  4B. 


CON-VERTED  CONTRACT  COST  DATA  USED  IN  HYPOTHESIS  TESTING — CONTRACTOR  #1 


Table  2B 

onverted  contract  cost  data  used  in  hypothesis  testing-contractor  #2 


CONVERTED  CONTRACT  COST  DATA  USED  IN  HYPOTHESIS  TESTING— CONTRACTOR  #3 


REVISED 

NEGOTIATED  ♦ • 306.0 

COMPLETED  - • • 259.5 


CONVERTED  CONTRACT  COST  DATA  USED  IN  HYPOTHESIS  TESTING — CONTRACTOR  #4 
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